Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple broker and coordinator interface #6675

Draft
wants to merge 25 commits into
base: main
Choose a base branch
from

Conversation

unkcpz
Copy link
Member

@unkcpz unkcpz commented Dec 20, 2024

  • all FIXME notes resolved
  • MockCoordinator -> InMemoryCoordinator and an exmaple of submit without rmq.
  • runner with dedicated thread (need more experiments)
  • Which change in plumpy/rmq-out resolve the test_disconnect timeout?
  • BroadcastFilter absorbed into add_broadcast_subscriber API.
  • broker and coordinator are duplicate abstraction, remove broker this should be done after, in this change, it bring coordinator layer first to not break the broker abstraction introduced (which is not real abstraction and relatively useless).
  • Move all kiwipy/rmq modules into broker/rabbitmq and make aiida-core not directly (but through broker/coordinator interface) dep on it.

@unkcpz unkcpz requested a review from agoscinski as a code owner December 20, 2024 15:46
@unkcpz unkcpz marked this pull request as draft December 20, 2024 15:46
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch 3 times, most recently from cff69d3 to b1f446a Compare December 20, 2024 17:42
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from afc0925 to 28cdb1c Compare December 21, 2024 02:08
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch 3 times, most recently from e613a3b to 5d59e6a Compare December 21, 2024 11:09
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from c6c6352 to c769906 Compare December 21, 2024 13:52
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from abaf3e9 to 03f7a5b Compare December 27, 2024 00:36
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from ca1dd09 to 02a939e Compare December 27, 2024 01:39
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from fe92318 to 329c51c Compare December 27, 2024 13:29
unkcpz and others added 3 commits December 27, 2024 14:30
When calling add_rpc_subscriber and add_task_subscriber, the event loop
of caller may from random event loop. But the target event loop is the
runner one. Therefore it requires to pass the loop to the broker when
creating the runner and runner's broker.
Instead of as a method of runner which is not needed and confuse.
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from 9583218 to 69db549 Compare December 28, 2024 00:35
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from a08471c to 5746ae8 Compare December 29, 2024 00:44
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from 5572660 to d62816e Compare December 29, 2024 22:59
@unkcpz unkcpz force-pushed the rmq-out-small-step-try branch from ccf4af2 to d62816e Compare December 30, 2024 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant